Defective pixel detector, imaging device, and defective pixel detection method

ABSTRACT

In a defective pixel detector  20  mounted on an imaging device  100 , a pixel data acquisition module  20   a  successively obtains pixel data of a target pixel set as an object of defective pixel detection and pixel data of plural surrounding pixels located in a neighborhood of the target pixel. A first operation module  20   b  calculates absolute values of differences between pixel data of multiple specific peripheral pixels selected among the plural surrounding pixels, as first absolute values. A defective pixel criterion setting module  20   c  sets a defective pixel criterion, based on differences between the multiple first absolute values and a preset threshold value. A second operation module  20   d  calculates absolute values of differences between the pixel data of the target pixel and the pixel data the multiple specific peripheral pixels, as second absolute values. A defective pixel identification module  20   e  identifies whether the target pixel is a defective pixel, based on the multiple second absolute values and the set defective pixel criterion. This arrangement enables successive and accurate detection of defective pixels in the imaging process.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a defective pixel detector mounted onan imaging device, and more specifically pertains to a technique ofdetecting defective pixels caused by, for example, malfunction of animage sensor included in the imaging device, among multiple pixelsconstituting an image taken by the imaging device.

2. Description of the Related Art

Various imaging devices including digital still cameras and digitalvideo cameras have become popular and been widely used. The imagingdevice is equipped with an image sensor designed to convert lightreceived via lenses into electrical signals. A CCD (charge coupleddevice) sensor and a CMOS (complementary metal oxide semiconductor)sensor are typical examples adopted for the image sensor. The imagesensor includes multiple light receiving elements (photodiodes) providedcorresponding to multiple pixels constituting a taken image and arrangedto respectively output pixel data representing pixel values of thecorresponding pixels. In display of an image taken with the imagingdevice on a display device, malfunction of any light receiving elementin the imaging process may cause output of pixel data having a higherpixel value than an originally expected pixel value. Such output causesa pixel corresponding to the malfunctioning light receiving element tobe recognized as a white defect. Malfunction of any light receivingelement in the imaging process may otherwise cause output of pixel datahaving a lower pixel value than the originally expected pixel value.Such output causes a pixel corresponding to the malfunctioning lightreceiving element to be recognized as a black defect.

Various techniques have been proposed for the imaging device to detectany defective pixel caused by malfunction of the image sensor amongmultiple pixels constituting a taken image and to correct pixel datarepresenting a pixel value of the detected defective pixel. A prior arttechnique disclosed in Japanese Patent Laid-Open No. 2001-86517 isapplicable to, for example, detect a white defect as a defective pixelin a single plate color video camera. This prior art technique checksthe absence of a high-frequency component in a target pixel set as aprocessing object based on the frequency characteristics of peripheralpixels with different color filters from the color filter set on thetarget pixel, and identifies the target pixel as a defective pixel inresponse to subsequent detection of the presence of the high frequencycomponent in the target pixel. Another prior art technique disclosed inJapanese Patent Laid-Open No. 2002-344814 successively stores 3×3 pixelblocks respectively including target pixels on their centers into abuffer in the imaging process, and compares the pixel value of thetarget pixel with the pixel value of each of multiple peripheral pixels.This prior art technique then counts a number Hn of peripheral pixelshaving the larger pixel values than the pixel value of the target pixeland a number Ln of peripheral pixels having the smaller pixel valuesthan the pixel value of the target pixel. When the counted number Hn ofthe peripheral pixels having the larger pixel values than the pixelvalue of the target pixel is equal to or greater than a value ‘5’, thetarget pixel is identified as a black defect. The pixel value of thetarget pixel is then replaced by an average of the peripheral pixelshaving the larger pixel values and is output as a corrected pixel value.When the counted number Ln of the peripheral pixels having the smallerpixel values than the pixel value of the target pixel is equal to orgreater than a value ‘5’, on the other hand, the target pixel isidentified as a white defect. The pixel value of the target pixel isthen replaced by an average of the peripheral pixels having the smallerpixel values and is output as a corrected pixel value.

The proposed prior art techniques of Japanese Patent Laid-Open No.2001-86517 and No. 2002-344814 are, however, not sufficient for accuratedetection of defective pixels. There is still room for furtherimprovement.

SUMMARY OF THE INVENTION

In a defective pixel detector mounted on an imaging device using animage sensor, there would thus be a demand for successive and accuratedetection of defective pixels in the imaging process.

The present invention accomplishes at least part of the demandsmentioned above by the following configurations applied to the defectivepixel detector, the imaging device, and the defective pixel detectionmethod.

According to one aspect, the present invention is directed to adefective pixel detector mounted on an imaging device and constructed todetect a defective pixel among multiple pixels constituting an imagetaken with the imaging device. The defective pixel detector includes: apixel data acquisition module configured to successively obtain pixeldata representing a pixel value of a target pixel set as an object ofdefective pixel detection and pixel data representing pixel values ofplural surrounding peripheral pixels located in a neighborhood of thetarget pixel; a first operation module configured to calculate absolutevalues of differences between pixel values of multiple specificperipheral pixels selected among the plural surrounding pixels, as firstabsolute values; a defective pixel criterion setting module configuredto set a defective pixel criterion, which is used in subsequentidentification of whether the target pixel is a defective pixel, basedon differences between the multiple first absolute values and a presetthreshold value; a second operation module configured to calculateabsolute values of differences between the pixel value of the targetpixel and the pixel values of the multiple specific peripheral pixels,as second absolute values; and a defective pixel identification moduleconfigured to identify whether the target pixel is a defective pixel,based on the multiple second absolute values and the set defective pixelcriterion.

The inventors of the present application have noted that pixel data ofperipheral pixels located in the neighborhood of a target pixel set asthe object of defective pixel detection affect the potential forrecognition of the target pixel as a defective pixel. In the case ofrelatively close pixel values of the peripheral pixels, the target pixelis likely to be recognized as a defective pixel even when the pixelvalue of the target pixel has relatively small differences from thepixel values of the peripheral pixels. In the case of relativelydiscrete pixel values of the peripheral pixels, on the other hand, thetarget pixel is unlikely to be recognized as a defective pixel even whenthe pixel value of the target pixel has relatively large differencesfrom the pixel values of the peripheral pixels. Such finding is true,irrespective of whether the defective pixel is a white defect or a blackdefect.

The defective pixel detector according to one aspect of the inventionsuccessively obtains the pixel data representing the pixel value of thetarget pixel set as the object of defective pixel detection and thepixel data representing the pixel values of the plural surroundingperipheral pixels located in the neighborhood of the target pixel. Thenumber of pixel data obtained may be set arbitrarily. For example, in amatrix arrangement of multiple light receiving elements included in animage sensor mounted on the imaging device, the defective pixel detectormay successively obtain pixel data of each 5×5 pixel block including atarget pixel on its center. The defective pixel detector subsequentlycalculates the absolute values of the differences between the pixelvalues of the multiple specific peripheral pixels selected among theplural surrounding pixels as the first absolute values, and sets thedefective pixel criterion, which is used in subsequent identification ofwhether the target pixel is a defective pixel, based on the differencesbetween the multiple first absolute values and the preset thresholdvalue. The multiple specific peripheral pixels and the preset thresholdvalue may be set arbitrarily according to the arrangement of the lightreceiving elements in the image sensor and according to the presence orthe absence of color filters set on the respective light receivingelements. This enables the defective pixel criterion to be setadequately according to the potential for recognition of the targetpixel as a defective pixel. The defective pixel detector then calculatesthe absolute values of the differences between the pixel value of thetarget pixel and the pixel values of the multiple specific peripheralpixels as the second absolute values, and identifies whether the targetpixel is a defective pixel, based on the multiple second absolute valuesand the set defective pixel criterion. The defective pixel detector ofthis arrangement mounted on the imaging device equipped with the imagesensor determines whether each of the target pixels has a high potentialfor recognition as a defective pixel or a low potential for recognitionas a defective pixel in the imaging process and successively andaccurately identifies whether each of the target pixels is a defectivepixel, based on the results of the determination.

A CCD and a CMOS sensor are typical examples adopted for the imagesensor. In the image sensor, the light receiving elements may be arrayedin a matrix or may be arrayed in a honeycomb structure.

In one preferable application of the defective pixel detector accordingto the above aspect of the invention, the defective pixel criterionsetting module sets the defective pixel criterion to a first defectivepixel criterion when the multiple first absolute values are respectivelynot less than the preset threshold value, while setting the defectivepixel criterion to a second defective pixel criterion, which is smallerthan the first defective pixel criterion, when the multiple firstabsolute values are respectively less than the preset threshold value.

In the defective pixel detector of this application, when the multiplefirst absolute values are respectively not less than the presetthreshold value, that is, in the case of relatively discrete pixelvalues of the multiple specific peripheral pixels, the defective pixelcriterion setting module determines that the target pixel has a lowpotential for recognition as a defective pixel and sets the defectivepixel criterion to the first defective pixel criterion. When themultiple first absolute values are respectively less than the presetthreshold value, that is, in the case of relatively close pixel valuesof the multiple specific peripheral pixels, on the other hand, thedefective pixel criterion setting module determines that the targetpixel has a high potential for recognition as a defective pixel and setsthe defective pixel criterion to the second defective pixel criterion,which is smaller than the first defective pixel criterion. In thedefective pixel detector of this application, the defective pixelcriterion setting module sets the stricter criterion of identifying thetarget pixel as a defective pixel for the target pixel likely to berecognized as a defective pixel than the criterion for the target pixelunlikely to be recognized as a defective pixel. This arrangement enablesthe stricter and thereby accurate identification of the target pixel asa defective pixel when the target pixel has a high potential forrecognition as a defective pixel.

In one concrete procedure of this application, the defective pixelcriterion setting module may set the defective pixel criterion to thefirst defective pixel criterion when at least one of the multiple firstabsolute values is not less than the preset threshold value, whilesetting the defective pixel criterion to the second defective pixelcriterion when all the multiple first absolute values are less than thepreset threshold value. In another concrete procedure of thisapplication, the defective pixel criterion setting module may set thedefective pixel criterion to the first defective pixel criterion whenall the multiple first absolute values are not less than the presetthreshold value, while setting the defective pixel criterion to thesecond defective pixel criterion when at least one of the multiple firstabsolute values is less than the preset threshold value.

In the defective pixel detector of this application, it is preferablethat the defective pixel criterion setting module sets at least one ofthe first defective pixel criterion and the second defective pixelcriterion, based on the pixel values of the multiple specific peripheralpixels.

The larger pixel values of the multiple specific peripheral pixelsrepresenting the higher luminance increase the potential for recognitionof the target pixel as a black defect. The smaller pixel values of themultiple specific peripheral pixels representing the lower luminanceincrease the potential for recognition of the target pixel as a whitedefect. In other words, the larger pixel values of the multiple specificperipheral pixels representing the higher luminance decrease thepotential for recognition of the target pixel as a white defect. Thesmaller pixel values of the multiple specific peripheral pixelsrepresenting the lower luminance decrease the potential for recognitionof the target pixel as a black defect.

In the defective pixel detector of this arrangement, the defective pixelcriterion setting module sets at least one of the first defective pixelcriterion and the second defective pixel criterion, based on the pixelvalues of the multiple specific peripheral pixels. The defective pixelidentification module can thus strictly identify whether the targetpixel is a defective pixel.

One concrete procedure of this arrangement sets at least one of thefirst defective pixel criterion and the second defective pixelcriterion, based on a simple average or a weighted average of the pixelvalues of the multiple specific peripheral pixels. In this case, theremay be a linear or non-linear relation between the defective pixelcriterion and the simple average or the weighted average of the pixelvalues of the multiple specific peripheral pixels.

In another preferable application of the defective pixel detectoraccording to the above aspect of the invention, the defective pixelidentification module identifies the target pixel as a defective pixelwhen the multiple second absolute values are respectively greater thanthe defective pixel criterion.

In the defective pixel detector of this application, the defective pixelidentification module can readily identify the target pixel as adefective pixel or as a non-defective pixel. In one concrete procedureof this application, the defective pixel identification moduleidentifies the target pixel as a defective pixel when at least one ofthe multiple second absolute values is greater than the defective pixelcriterion. In another concrete procedure of this application, thedefective pixel identification module identifies the target pixel as adefective pixel when all the multiple second absolute values are greaterthan the defective pixel criterion.

In one preferable embodiment of the defective pixel detector of theinvention, the imaging device is equipped with an image sensor includingmultiple light receiving elements provided corresponding to the multiplepixels and arranged to respectively output pixel data representing pixelvalues of the corresponding pixels. Multiple different types of colorfilters designed to transmit different color lights are set in apredetermined arrangement on the multiple light receiving elements. Themultiple specific peripheral pixels correspond to specific lightreceiving elements having a selected type of color filters identicalwith a color filter set on a light receiving element corresponding tothe target pixel.

In the defective pixel detector of this embodiment having the multipledifferent types of color filters designed to transmit different colorlights and set in the predetermined arrangement on the multiple lightreceiving elements, the defective pixel identification module enablesstricter and thereby accurate identification of the target pixel as adefective pixel or a non-defective pixel.

In the defective pixel detector of the above embodiment, the presetthreshold value may be set for each of the multiple different types ofcolor filters.

In the defective pixel detector of the embodiment having the multipledifferent types of color filters designed to transmit different colorlights and set in the predetermined arrangement on the multiple lightreceiving elements, the defective pixel identification module enablesstricter and thereby accurate identification of the target pixel as adefective pixel or a non-defective pixel according to the type of thecolor filter.

In the defective pixel detector of the above embodiment, the defectivepixel criterion may be set for each of the multiple different types ofcolor filters.

In the defective pixel detector of the embodiment having the multipledifferent types of color filters designed to transmit different colorlights and set in the predetermined arrangement on the multiple lightreceiving elements, the defective pixel identification module enablesstricter and thereby accurate identification of the target pixel as adefective pixel or a non-defective pixel according to the type of thecolor filter.

According to another aspect, the invention is directed to an imagingdevice including: an imaging assembly equipped with an image sensorincluding multiple light receiving elements provided corresponding tomultiple pixels constituting a taken image and arranged to respectivelyoutput pixel data representing pixel values of the corresponding pixels;a defective pixel detector configured to detect a defective pixel amongthe multiple pixels, based on the pixel data respectively output fromthe multiple light receiving elements; and a pixel data correction unitconfigured to correct pixel data representing a pixel value of thedetected defective pixel. The defective pixel detector included in thisimaging device may be configured to have any of the arrangementsdescribed above.

When the image taken with the imaging device has defective pixels, theimaging device of this arrangement successively and accurately detectsthe defective pixels in the imaging process and adequately corrects thepixel data of the detected defective pixels.

In one preferable application of the imaging device according to theabove aspect of the invention, the pixel data correction unit replacesthe pixel data of the detected defective pixel by an average of thepixel values of the multiple specific peripheral pixels.

In the imaging device of this application, the pixel data correctionunit readily corrects the pixel data of the detected defective pixel.The correction of the pixel data of the defective pixel may replace thepixel data of the defective pixel by a weighted average of the pixelvalues of the multiple specific peripheral pixels, instead of the simpleaverage of the pixel values of the multiple specific peripheral pixels.

The present invention is not restricted to the defective pixel detectoror the imaging device described above, but may be actualized bydiversity of other applications, for example, a defective pixeldetection method, a computer program for actualizing any of thedefective pixel detector, the imaging device, and the defective pixeldetection method, a recording medium in which such a computer program isrecorded, and a data signal configured to include such a computerprogram and embodied in a carrier wave. Any of the various additionalarrangements explained above may be adopted for any of theseapplications.

In the applications of the invention as the computer program and therecording medium in which the computer program is recorded, theinvention may be given as a whole program to control the operations ofthe defective pixel detector or the imaging device or as a partialprogram to exert only the characteristic functions of the invention.Available examples of the recording medium include flexible disks,CD-ROMs, DVD-ROMs, magneto-optical disks, IC cards, ROM cartridges,punched cards, prints with barcodes or other codes printed thereon,internal storage devices (memories like RAMs and ROMs) and externalstorage devices of the computer, and diversity of other computerreadable media.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates the structure of an imaging device in afirst embodiment of the invention;

FIG. 2 shows an array of light receiving elements in an image sensorincluded in an imaging assembly of the imaging device;

FIG. 3 is an explanatory view schematically showing the outline of adefective pixel detection process and a pixel data correction processperformed in the first embodiment;

FIG. 4 is a flowchart showing the details of the defective pixeldetection process and the pixel data correction process in the firstembodiment;

FIG. 5 is a flowchart showing the details of the defective pixeldetection process and the pixel data correction process in the firstembodiment;

FIG. 6 is a flowchart showing the details of the defective pixeldetection process and the pixel data correction process in the firstembodiment;

FIG. 7 is a flowchart showing the details of the defective pixeldetection process and the pixel data correction process in the firstembodiment;

FIGS. 8( a) through (c) schematically show arrangements of pixels in a5×5 pixel block;

FIG. 9 schematically illustrates the structure of another imaging devicein a second embodiment of the invention;

FIG. 10 is a flowchart showing a defective pixel detection process and apixel data correction process performed in the second embodiment; and

FIG. 11 schematically shows an arrangement of pixels in a 3×3 pixelblock.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Some modes of carrying out the invention are described below in thefollowing sequence as preferred embodiments with reference to theaccompanied drawings:

A. First Embodiment

A1. Structure of Imaging Device

A2. Defective Pixel Detection Process and Pixel Data Correction Process

-   -   A2.1. Red (R) Target Pixel    -   A2.2. Green (G) Target Pixel    -   A2.3. Blue (B) Target Pixel

B. Second Embodiment

-   -   B1. Structure of Imaging Device    -   B2. Defective Pixel Detection Process and Pixel Data Correction        Process

C. Other Aspects A. First Embodiment A1. Structure of Imaging Device

FIG. 1 schematically illustrates the structure of an imaging device 100in a first embodiment of the invention. The imaging device 100 includesan imaging assembly 10, a defective pixel detector 20, a pixel datacorrection unit 30, and an output unit 40. The imaging device 100successively detects defective pixels (white defects and black defects)among multiple pixels constituting an image taken with the imagingassembly 10, corrects pixel data of the detected defective pixels, andoutputs the corrected pixel data as explained below. The defective pixeldetector 20, the pixel data correction unit 30, and the output unit 40are configured by the hardware in this embodiment, although some part ofthese constituents may be implemented by the software configuration. Theimaging device 100 also includes a display unit constructed to displaythe taken image, for example, a liquid crystal panel, and a recorderunit constructed to store the taken image as pixel data in a recordingmedium, such as a flash memory, although not being specificallyillustrated. The imaging device 100 further has a control unit 50configured to include a CPU, a RAM, and a ROM and control the respectiveconstituents of the imaging device 100.

The imaging assembly 10 has a zoom lens, a focusing lens, and anaperture mechanism (not shown), as well as an image sensor 12 arrangedto convert the light received via these elements into electricalsignals. The image sensor 12 adopted in this embodiment is a CMOS sensorand includes multiple light receiving elements (photodiodes) providedcorresponding to the multiple pixels constituting the taken image tooutput pixel data representing pixel values of the respectivelycorresponding pixels.

FIG. 2 shows an array of light receiving elements 12 d in the imagesensor 12. The imaging device 100 of the embodiment has the imagingassembly 10 of a single-plate type including only one image sensor 12.The image sensor 12 of this embodiment has multiple light receivingelements 12 d arrayed in a matrix. The multiple light receiving elements12 d in the image sensor 12 may otherwise be arrayed in a honeycombstructure. Each of the light receiving elements 12 d has one colorfilter. In the structure of this embodiment, the respective lightreceiving elements 12 d have color filters of three primary colors, red(R), green (G), and blue (B) arrayed as shown in FIG. 2. In anotherexample, the respective light receiving elements 12 d may have colorfilters of three complementary colors, cyan, magenta, and yellow in apreset arrangement. The number of the light receiving elements 12 dincluded in the image sensor 12 is determined arbitrarily correspondingto an optical resolution demand.

In the imaging device 100 of FIG. 1, the defective pixel detector 20includes a pixel data acquisition module 20 a, a first computationmodule 20 b, a defective pixel criterion setting module 20 c, a secondcomputation module 20 d, and a defective pixel identification module 20e. The defective pixel detector 20 functions to detect defective pixelsamong multiple pixels constituting an image taken with the imagingassembly 10. The pixel data acquisition module 20 a has a buffer 20 ab.The defective pixel detector 20 corresponds to the defective pixeldetector of the invention. The following describes the functions of therespective constituents of the defective pixel detector 20, a defectivepixel detection process, and a pixel data correction process performedby the pixel data correction unit 30.

A2. Defective Pixel Detection Process and Pixel Data Correction Process

The description sequentially regards the outline and the details of thedefective pixel detection process and the pixel data correction process.

FIG. 3 is an explanatory view schematically showing the outline of thedefective pixel detection process and the pixel data correction processperformed in the first embodiment. In the procedure of this embodiment,a first step (1) obtains pixel data of each 5×5 pixel block PB(i,j) as acurrent processing object encircled by the broken line, out of multiplepixels constituting a taken image. In the pixel block PB(i,j), ‘i’ and‘j’ respectively represent an i-th pixel block rightward from an upperleft end of the image and a j-th pixel block downward from the upperleft end of the image. A second step (2) determines whether a hatchedtarget pixel as an object of defective pixel detection located on thecenter of the object pixel block PB(i,j) has a high potential forrecognition as a defective pixel, based on pixel values of peripheralpixels located in the neighborhood of the target pixel. A third step (3)sets a criterion for identification of the target pixel as a defectivepixel according to the result of the determination. A fourth step (4)identifies whether the target pixel is a defective pixel, based on theset criterion. A fifth step (5) corrects the pixel value of the targetpixel, when the target pixel is identified as a defective pixel. Thisseries of processing is repeatedly executed with a shift of the objectpixel block in a sequence of a pixel block PB(1,1), a pixel blockPB(2,1), a pixel block PB(3,1) . . . , a pixel block PB(1,2) . . . , toa last pixel block including a last target pixel as shown in FIG. 3.

FIGS. 4 through 7 are flowcharts showing the details of the defectivepixel detection process and the pixel data correction process in thefirst embodiment. FIGS. 8( a) through (c) schematically showarrangements of pixels in the 5×5 pixel block. FIGS. 8( a), 8(b), and8(c) respectively show arrangements of pixels in the 5×5 pixel blockwith regard to the red (R) target pixel, the green (G) target pixel, andthe blue (B) target pixel. In the respective arrangements of the 5×5pixel block shown in FIGS. 8( a) through 8(c), a pixel located on thecenter of the pixel block and encircled by the thick line represents atarget pixel as an object of defective pixel detection. Amongsurrounding pixels located in the neighborhood of the target pixel,hatched pixels represent peripheral pixels for defective pixelidentification as explained later.

Referring to the flowchart of FIG. 4, the pixel data acquisition module20 a first obtains analog pixel data of each 5×5 pixel block as acurrent processing object out of analog pixel data output from therespective light receiving elements 12 d of the image sensor 12,performs analog-to-digital (A-D) conversion of the obtained analog pixeldata, stores the A-D converted pixel data as pixel values of therespective pixels in the object pixel block into the buffer 20 ab, andsets a center pixel located on the center of the object pixel block as atarget pixel (step S100). The pixel data acquisition module 20 asubsequently identifies the color of the set target pixel or the colorof the color filter set on a specific light receiving element 12 dcorresponding to the target pixel among red (R), green (G), and blue (B)(step S110). In the image sensor 12, the array of the color filtersrespectively set on the multiple light receiving elements 12 d isdetermined in advance. The color of the target pixel is thus readilyidentifiable by counting the number of object pixel blocks from thestart of the processing.

The pixel data acquisition module 20 a selects pixels corresponding tothe light receiving elements 12 d having the same color filter (the samecolor) as the color filter (color) of the specific light receivingelement 12 d corresponding to the target pixel, among surrounding pixelslocated in the neighborhood of the target pixel and specifies theselected pixels as peripheral pixels for defective pixel identificationused for identification of the target pixel as a defective pixel. Uponidentification of the color of the target pixel as red (R) at step S110,the pixel data acquisition module 20 a specifies surrounding pixels R1to R8 as peripheral pixels for defective pixel identification as shownin FIG. 8( a) (step S120). Upon identification of the color of thetarget pixel as green (G) at step S110, the pixel data acquisitionmodule 20 a specifies surrounding pixels G1 to G12 as peripheral pixelsfor defective pixel identification as shown in FIG. 8( b) (step S130).Upon identification of the color of the target pixel as blue (B) at stepS110, the pixel data acquisition module 20 a specifies surroundingpixels B1 to B8 as peripheral pixels for defective pixel identificationas shown in FIG. 8( c) (step S140). In this embodiment, all the hatchedsurrounding pixels are specified as the peripheral pixels for defectivepixel identification. The peripheral pixels for defective pixelidentification may, however, be specified arbitrarily according to thearray of the light receiving elements 12 d in the image sensor 12 andaccording to the array of the color filters set on the respective lightreceiving elements 12 d.

A2.1. Red (R) Target Pixel

Upon identification of the color of the target pixel as red (R) at stepS110 in the flowchart of FIG. 4, the processing flow proceeds to theflowchart of FIG. 5 after the specification of the peripheral pixels fordefective pixel identification at step S120. The first computationmodule 20 b calculates absolute values Vabs1(R) of respectivedifferences between the pixel values of the peripheral pixels R1 to R8for defective pixel identification specified with regard to the red (R)target pixel Rt as shown in FIG. 8( a) (step S200). The absolute valuesVabs1(R) are used for subsequent determination of whether the targetpixel Rt has a high potential for recognition as a defective pixel andare referred to by the defective pixel criterion setting module 20 c toset a defective pixel criterion Vjth(R) for the red (R) target pixel Rtas explained later. The absolute values Vabs1(R) correspond to the firstabsolute values of the invention.

In this embodiment, the absolute values Vabs1(R) of the differencescalculated by the first computation module 20 b include the absolutevalue of the difference between the pixel values of the peripheralpixels R1 and R2 for defective pixel identification, the absolute valueof the difference between the pixel values of the peripheral pixels R2and R3 for defective pixel identification, the absolute value of thedifference between the pixel values of the peripheral pixels R3 and R5for defective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels R5 and R8 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels R8 and R7 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels R7 and R6 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels R6 and R4 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels R4 and R5 fordefective pixel identification, and the absolute value of the differencebetween the pixel values of the peripheral pixels R2 and R7 fordefective pixel identification (see FIG. 8( a)).

The defective pixel criterion setting module 20 c determines whether atleast one of the multiple calculated absolute values Vabs1(R) is notless than a threshold value Vth(R) of the red (R) target pixel Rt (stepS210). This determines whether the target pixel Rt has a high potentialfor recognition as a defective pixel. When at least one of the multipleabsolute values Vabs1(R) is not less than the threshold value Vth(R), itis determined that the target pixel Rt has a low potential forrecognition as a defective pixel. When all the multiple absolute valuesVabs1(R) are less than the threshold value Vth(R), on the other hand, itis determined that the target pixel Rt has a high potential forrecognition as a defective pixel. Such determination is based on thefindings that relatively discrete pixel values of the peripheral pixelsfor defective pixel identification make the target pixel unlikely to berecognized as a defective pixel and that relatively close pixel valuesof the peripheral pixels for defective pixel identification make thetarget pixel likely to be recognized as a defective pixel. The thresholdvalue Vth(R) corresponds to the preset threshold value of the invention.

When at least one of the multiple absolute values Vabs1(R) is not lessthan the threshold value Vth(R) (step S210: Yes), the defective pixelcriterion setting module 20 c sets a defective pixel criterion Vjth(R)for the red (R) target pixel Rt, which will be used in a later step bythe defective pixel identification module 20 e, to a preset thresholdvalue Vjth1(R) (Vjth(R)=Vjth1(R)) (step S211). The threshold valueVjth1(R) corresponds to the first defective pixel criterion of theinvention.

The second computation module 20 d subsequently calculates absolutevalues Vabs2(R) of differences between the pixel value of the targetpixel Rt and the pixel values of the peripheral pixels R1 to R8 fordefective pixel identification specified with regard to the target pixelRt (step S212). The absolute values Vabs2(R) are used for subsequentidentification of whether the target pixel Rt is a defective pixel bythe defective pixel identification module 20 e as explained later. Theabsolute values Vabs2(R) correspond to the second absolute values of theinvention.

The defective pixel identification module 20 e then determines whetherall the multiple calculated absolute values Vabs2(R) are greater thanthe threshold value Vjth1(R) (step S213). When all the multiple absolutevalues Vabs2(R) are greater than the threshold value Vjth1(R) (stepS213: Yes), the defective pixel identification module 20 e identifiesthe target pixel Rt as a defective pixel (step S220).

The pixel data correction unit 30 then corrects the pixel data of thetarget pixel Rt (step S230). According to the concrete procedure of thisembodiment, the pixel data correction unit 30 obtains the pixel valuesof the peripheral pixels R1 to R8 for defective pixel identificationspecified with regard to the target pixel Rt from the buffer 20 ab,calculates an average of the obtained pixel values, and replaces thepixel value of the target pixel Rt by the calculated average to correctthe pixel data of the pixel value Rt. The output unit 40 outputs thecorrected pixel data (step S240).

When at least one of the multiple absolute values Vabs2(R) is notgreater than the threshold value Vjth1(R) (step S213: No), on the otherhand, the defective pixel identification module 20 e identifies thetarget pixel Rt as no defective pixel. Without any correction of thepixel data of the target pixel Rt by the pixel data correction unit 30,the output unit 40 outputs the pixel data (step S240).

Upon completion of the processing and the output of pixel data withregard to all the pixels (step S250: yes), the processing flowterminates the defective pixel detection process and the pixel datacorrection process. Upon no completion of the processing and the outputof pixel data with regard to all the pixels (step S250: No), on theother hand, the processing flow goes back to step S100 in the flowchartof FIG. 4 to detect a defective pixel with regard to a target pixel of anext object pixel block.

When all the multiple absolute values Vabs1(R) are less than thethreshold value Vth(R) (step S210: No), the defective pixel criterionsetting module 20 c sets the defective pixel criterion Vjth(R) for thered (R) target pixel Rt, which will be used in a later step by thedefective pixel identification module 20 e, to a preset threshold valueVjth2(R) (Vjth(R)=Vjth2(R)) (step S214). The threshold value Vjth2(R) issmaller than the threshold value Vjth1(R). This sets the strictercriterion for identification of the target pixel Rt as a defectivepixel, since the determination result of step S210 that all the multipleabsolute values Vabs1(R) are less than the threshold value Vth(R)suggests that the target pixel Rt has a high potential for recognitionas a defective pixel. The threshold value Vjth2(R) corresponds to thesecond defective pixel criterion of the invention.

In the same manner as the calculation at step S212, the secondcomputation module 20 d calculates the absolute values Vabs2(R) of thedifferences between the pixel value of the target pixel Rt and the pixelvalues of the peripheral pixels R1 to R8 for defective pixelidentification specified with regard to the target pixel Rt (step S215).

The defective pixel identification module 20 e then determines whetherall the multiple calculated absolute values Vabs2(R) are greater thanthe threshold value Vjth2(R) (step S216). When all the multiple absolutevalues Vabs2(R) are greater than the threshold value Vjth2(R) (stepS216: Yes), the defective pixel identification module 20 e identifiesthe target pixel Rt as a defective pixel (step S220).

The pixel data correction unit 30 then corrects the pixel data of thetarget pixel Rt according to the procedure described above (step S230),and the output unit 40 outputs the corrected pixel data (step S240).

When at least one of the multiple absolute values Vabs2(R) is notgreater than the threshold value Vjth2(R) (step S216: No), on the otherhand, the defective pixel identification module 20 e identifies thetarget pixel Rt as no defective pixel. Without any correction of thepixel data of the target pixel Rt by the pixel data correction unit 30,the output unit 40 outputs the pixel data (step S240).

Upon completion of the processing and the output of pixel data withregard to all the pixels (step S250: yes), the processing flowterminates the defective pixel detection process and the pixel datacorrection process. Upon no completion of the processing and the outputof pixel data with regard to all the pixels (step S250: No), on theother hand, the processing flow goes back to step S100 in the flowchartof FIG. 4 to detect a defective pixel with regard to a target pixel of anext object pixel block.

The defective pixel detection process and the pixel data correctionprocess for the red (R) target pixel Rt described above with referenceto the flowchart of FIG. 5 are similarly performed for the green (G)target pixel Gt and the blue (B) target pixel Bt as explained below.

A2.2. Green (G) Target Pixel

Upon identification of the color of the target pixel as green (G) atstep S110 in the flowchart of FIG. 4, the processing flow proceeds tothe flowchart of FIG. 6 after the specification of the peripheral pixelsfor defective pixel identification at step S130. The first computationmodule 20 b calculates absolute values Vabs1(G) of respectivedifferences between the pixel values of the peripheral pixels G1 to G12for defective pixel identification specified with regard to the green(G) target pixel Gt as shown in FIG. 8( b) (step S300). The absolutevalues Vabs1(G) are used for subsequent determination of whether thetarget pixel Gt has a high potential for recognition as a defectivepixel and are referred to by the defective pixel criterion settingmodule 20 c to set a defective pixel criterion Vjth(G) for the green (G)target pixel Gt as explained later. The absolute values Vabs1(G)correspond to the first absolute values of the invention.

In this embodiment, the absolute values Vabs1(G) of the differencescalculated by the first computation module 20 b include the absolutevalue of the difference between the pixel values of the peripheralpixels G1 and G2 for defective pixel identification, the absolute valueof the difference between the pixel values of the peripheral pixels G2and G3 for defective pixel identification, the absolute value of thedifference between the pixel values of the peripheral pixels G3 and G7for defective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G7 and G12 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G12 and G11 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G11 and G10 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G10 and G6 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G6 and G7 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G2 and G11 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G4 and G5 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G5 and G9 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G9 and G8 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G8 and G4 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels G4 and G9 fordefective pixel identification, and the absolute value of the differencebetween the pixel values of the peripheral pixels G5 and G8 fordefective pixel identification (see FIG. 8( b)).

The defective pixel criterion setting module 20 c determines whether atleast one of the multiple calculated absolute values Vabs1(G) is notless than a threshold value Vth(G) of the green (G) target pixel Gt(step S310). This determines whether the target pixel Gt has a highpotential for recognition as a defective pixel. When at least one of themultiple absolute values Vabs1(G) is not less than the threshold valueVth(G), it is determined that the target pixel Gt has a low potentialfor recognition as a defective pixel. When all the multiple absolutevalues Vabs1(G) are less than the threshold value Vth(G), on the otherhand, it is determined that the target pixel Gt has a high potential forrecognition as a defective pixel. As mentioned previously, suchdetermination is based on the findings that relatively discrete pixelvalues of the peripheral pixels for defective pixel identification makethe target pixel unlikely to be recognized as a defective pixel and thatrelatively close pixel values of the peripheral pixels for defectivepixel identification make the target pixel likely to be recognized as adefective pixel. The threshold value Vth(G) corresponds to the presetthreshold value of the invention.

When at least one of the multiple absolute values Vabs1(G) is not lessthan the threshold value Vth(G) (step S310: Yes), the defective pixelcriterion setting module 20 c sets a defective pixel criterion Vjth(G)for the green (G) target pixel Gt, which will be used in a later step bythe defective pixel identification module 20 e, to a preset thresholdvalue Vjth1(G) (Vjth(G)=Vjth1(G)) (step S311). The threshold valueVjth1(G) corresponds to the first defective pixel criterion of theinvention.

The second computation module 20 d subsequently calculates absolutevalues Vabs2(G) of differences between the pixel value of the targetpixel Gt and the pixel values of eight peripheral pixels G2, G4, G5, G6,G7, G8, G9, and G11 for defective pixel identification selected amongthe twelve peripheral pixels G1 to G12 for defective pixelidentification specified with regard to the target pixel Gt (step S312).These eight peripheral pixels G2, G4, G5, G6, G7, G8, G9, and G11 arerelatively close in distance from the target pixel Gt and are shown byright-down hatched rectangles in FIG. 8( b). The absolute valuesVabs2(G) are used for subsequent identification of whether the targetpixel Gt is a defective pixel by the defective pixel identificationmodule 20 e as explained later. The absolute values Vabs2(G) correspondto the second absolute values of the invention.

The defective pixel identification module 20 e then determines whetherall the multiple calculated absolute values Vabs2(G) are greater thanthe threshold value Vjth1(G) (step S313). When all the multiple absolutevalues Vabs2(G) are greater than the threshold value Vjth1(G) (stepS313: Yes), the defective pixel identification module 20 e identifiesthe target pixel Gt as a defective pixel (step S320).

The pixel data correction unit 30 then corrects the pixel data of thetarget pixel Gt (step S330). According to the concrete procedure of thisembodiment, the pixel data correction unit 30 obtains the pixel valuesof the eight right-down hatched peripheral pixels G2, G4, G5, G6, G7,G8, G9, and G11 for defective pixel identification from the buffer 20ab, calculates an average of the obtained pixel values, and replaces thepixel value of the target pixel Gt by the calculated average to correctthe pixel data of the pixel value Gt. The output unit 40 outputs thecorrected pixel data (step S340). These eight peripheral pixels G2, G4,G5, G6, G7, G8, G9, and G11 are relatively close in distance from thetarget pixel Gt and are selected among the twelve peripheral pixels G1to G12 for defective pixel identification specified with regard to thetarget pixel Gt as mentioned above.

When at least one of the multiple absolute values Vabs2(G) is notgreater than the threshold value Vjth1(G) (step S313: No), on the otherhand, the defective pixel identification module 20 e identifies thetarget pixel Gt as no defective pixel. Without any correction of thepixel data of the target pixel Gt by the pixel data correction unit 30,the output unit 40 outputs the pixel data (step S340).

Upon completion of the processing and the output of pixel data withregard to all the pixels (step S350: yes), the processing flowterminates the defective pixel detection process and the pixel datacorrection process. Upon no completion of the processing and the outputof pixel data with regard to all the pixels (step S350: No), on theother hand, the processing flow goes back to step S100 in the flowchartof FIG. 4 to detect a defective pixel with regard to a target pixel of anext object pixel block.

When all the multiple absolute values Vabs1(G) are less than thethreshold value Vth(G) (step S310: No), the defective pixel criterionsetting module 20 c sets the defective pixel criterion Vjth(G) for thegreen (G) target pixel Gt, which will be used in a later step by thedefective pixel identification module 20 e, to a preset threshold valueVjth2(G) (Vjth(G)=Vjth2(G)) (step S314). The threshold value Vjth2(G) issmaller than the threshold value Vjth1(G). This sets the strictercriterion for identification of the target pixel Gt as a defectivepixel, since the determination result of step S310 that all the multipleabsolute values Vabs1(G) are less than the threshold value Vth(G)suggests that the target pixel Gt has a high potential for recognitionas a defective pixel. The threshold value Vjth2(G) corresponds to thesecond defective pixel criterion of the invention.

In the same manner as the calculation at step S312, the secondcomputation module 20 d calculates the absolute values Vabs2(G) of thedifferences between the pixel value of the target pixel Gt and the pixelvalues of the eight right-down hatched peripheral pixels G2, G4, G5, G6,G7, G8, G9, and G11 for defective pixel identification (step S315).These eight peripheral pixels G2, G4, G5, G6, G7, G8, G9, and G11 arerelatively close in distance from the target pixel Gt and are selectedamong the twelve peripheral pixels G1 to G12 for defective pixelidentification specified with regard to the target pixel Gt as mentionedabove.

The defective pixel identification module 20 e then determines whetherall the multiple calculated absolute values Vabs2(G) are greater thanthe threshold value Vjth2(G) (step S316). When all the multiple absolutevalues Vabs2(G) are greater than the threshold value Vjth2(G) (stepS316: Yes), the defective pixel identification module 20 e identifiesthe target pixel Gt as a defective pixel (step S320).

The pixel data correction unit 30 then corrects the pixel data of thetarget pixel Gt according to the procedure described above (step S330),and the output unit 40 outputs the corrected pixel data (step S340).

When at least one of the multiple absolute values Vabs2(G) is notgreater than the threshold value Vjth2(G) (step S316: No), on the otherhand, the defective pixel identification module 20 e identifies thetarget pixel Gt as no defective pixel. Without any correction of thepixel data of the target pixel Gt by the pixel data correction unit 30,the output unit 40 outputs the pixel data (step S340).

Upon completion of the processing and the output of pixel data withregard to all the pixels (step S350: yes), the processing flowterminates the defective pixel detection process and the pixel datacorrection process. Upon no completion of the processing and the outputof pixel data with regard to all the pixels (step S350: No), on theother hand, the processing flow goes back to step S100 in the flowchartof FIG. 4 to detect a defective pixel with regard to a target pixel of anext object pixel block.

A2.3. Blue (B) Target Pixel

Upon identification of the color of the target pixel as blue (B) at stepS110 in the flowchart of FIG. 4, the processing flow proceeds to theflowchart of FIG. 7 after the specification of the peripheral pixels fordefective pixel identification at step S140. The first computationmodule 20 b calculates absolute values Vabs1(B) of respectivedifferences between the pixel values of the peripheral pixels B1 to B8for defective pixel identification specified with regard to the blue (B)target pixel Bt as shown in FIG. 8( c) (step 400). The absolute valuesVabs1(B) are used for subsequent determination of whether the targetpixel Bt has a high potential for recognition as a defective pixel andare referred to by the defective pixel criterion setting module 20 c toset a defective pixel criterion Vjth(B) for the blue (B) target pixel Btas explained later. The absolute values Vabs1(B) correspond to the firstabsolute values of the invention.

In this embodiment, the absolute values Vabs1(B) of the differencescalculated by the first computation module 20 b include the absolutevalue of the difference between the pixel values of the peripheralpixels B1 and B2 for defective pixel identification, the absolute valueof the difference between the pixel values of the peripheral pixels B2and B3 for defective pixel identification, the absolute value of thedifference between the pixel values of the peripheral pixels B3 and B5for defective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels B5 and B8 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels B8 and B7 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels B7 and B6 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels B6 and B4 fordefective pixel identification, the absolute value of the differencebetween the pixel values of the peripheral pixels B4 and B5 fordefective pixel identification, and the absolute value of the differencebetween the pixel values of the peripheral pixels B2 and B7 fordefective pixel identification (see FIG. 8( c)).

The defective pixel criterion setting module 20 c determines whether atleast one of the multiple calculated absolute values Vabs1(B) is notless than a threshold value Vth(B) of the blue (B) target pixel Bt (stepS410). This determines whether the target pixel Bt has a high potentialfor recognition as a defective pixel. When at least one of the multipleabsolute values Vabs1(B) is not less than the threshold value Vth(B), itis determined that the target pixel Bt has a low potential forrecognition as a defective pixel. When all the multiple absolute valuesVabs1(B) are less than the threshold value Vth(B), on the other hand, itis determined that the target pixel Bt has a high potential forrecognition as a defective pixel. As mentioned previously, suchdetermination is based on the findings that relatively discrete pixelvalues of the peripheral pixels for defective pixel identification makethe target pixel unlikely to be recognized as a defective pixel and thatrelatively close pixel values of the peripheral pixels for defectivepixel identification make the target pixel likely to be recognized as adefective pixel. The threshold value Vth(B) corresponds to the presetthreshold value of the invention.

When at least one of the multiple absolute values Vabs1(B) is not lessthan the threshold value Vth(B) (step S410: Yes), the defective pixelcriterion setting module 20 c sets a defective pixel criterion Vjth(B)for the blue (B) target pixel Bt, which will be used in a later step bythe defective pixel identification module 20 e, to a preset thresholdvalue Vjth1(B) (Vjth(B)=Vjth1(B)) (step S411). The threshold valueVjth1(B) corresponds to the first defective pixel criterion of theinvention.

The second computation module 20 d subsequently calculates absolutevalues Vabs2(B) of differences between the pixel value of the targetpixel Bt and the pixel values of the peripheral pixels B1 to B8 fordefective pixel identification specified with regard to the target pixelBt (step S412). The absolute values Vabs2(B) are used for subsequentidentification of whether the target pixel Bt is a defective pixel bythe defective pixel identification module 20 e as explained later. Theabsolute values Vabs2(B) correspond to the second absolute values of theinvention.

The defective pixel identification module 20 e then determines whetherall the multiple calculated absolute values Vabs2(B) are greater thanthe threshold value Vjth1(B) (step S413). When all the multiple absolutevalues Vabs2(B) are greater than the threshold value Vjth1(B) (stepS413: Yes), the defective pixel identification module 20 e identifiesthe target pixel Bt as a defective pixel (step S420).

The pixel data correction unit 30 then corrects the pixel data of thetarget pixel Bt (step S430). According to the concrete procedure of thisembodiment, the pixel data correction unit 30 obtains the pixel valuesof the peripheral pixels B1 to B8 for defective pixel identificationspecified with regard to the target pixel Bt from the buffer 20 ab,calculates an average of the obtained pixel values, and replaces thepixel value of the target pixel Bt by the calculated average to correctthe pixel data of the pixel value Bt. The output unit 40 outputs thecorrected pixel data (step S440).

When at least one of the multiple absolute values Vabs2(B) is notgreater than the threshold value Vjth1(B) (step S413: No), on the otherhand, the defective pixel identification module 20 e identifies thetarget pixel Bt as no defective pixel. Without any correction of thepixel data of the target pixel Bt by the pixel data correction unit 30,the output unit 40 outputs the pixel data (step S440).

Upon completion of the processing and the output of pixel data withregard to all the pixels (step S450: yes), the processing flowterminates the defective pixel detection process and the pixel datacorrection process. Upon no completion of the processing and the outputof pixel data with regard to all the pixels (step S450: No), on theother hand, the processing flow goes back to step S100 in the flowchartof FIG. 4 to detect a defective pixel with regard to a target pixel of anext object pixel block.

When all the multiple absolute values Vabs1(B) are less than thethreshold value Vth(B) (step S410: No), the defective pixel criterionsetting module 20 c sets the defective pixel criterion Vjth(B) for theblue (B) target pixel Bt, which will be used in a later step by thedefective pixel identification module 20 e, to a preset threshold valueVjth2(B) (Vjth(B)=Vjth2(B)) (step S414). The threshold value Vjth2(B) issmaller than the threshold value Vjth1(B). This sets the strictercriterion for identification of the target pixel Bt as a defectivepixel, since the determination result of step S410 that all the multipleabsolute values Vabs1(B) are less than the threshold value Vth(B)suggests that the target pixel Bt has a high potential for recognitionas a defective pixel. The threshold value Vjth2(B) corresponds to thesecond defective pixel criterion of the invention.

In the same manner as the calculation at step S412, the secondcomputation module 20 d calculates the absolute values Vabs2(B) of thedifferences between the pixel value of the target pixel Bt and the pixelvalues of the peripheral pixels B1 to B8 for defective pixelidentification specified with regard to the target pixel Bt (step S415).

The defective pixel identification module 20 e then determines whetherall the multiple calculated absolute values Vabs2(B) are greater thanthe threshold value Vjth2(B) (step S416). When all the multiple absolutevalues Vabs2(B) are greater than the threshold value Vjth2(B) (stepS416: Yes), the defective pixel identification module 20 e identifiesthe target pixel Bt as a defective pixel (step S420).

The pixel data correction unit 30 then corrects the pixel data of thetarget pixel Bt according to the procedure described above (step S430),and the output unit 40 outputs the corrected pixel data (step S440).

When at least one of the multiple absolute values Vabs2(B) is notgreater than the threshold value Vjth2(B) (step S416: No), on the otherhand, the defective pixel identification module 20 e identifies thetarget pixel Bt as no defective pixel. Without any correction of thepixel data of the target pixel Bt by the pixel data correction unit 30,the output unit 40 outputs the pixel data (step S440).

Upon completion of the processing and the output of pixel data withregard to all the pixels (step S450: yes), the processing flowterminates the defective pixel detection process and the pixel datacorrection process. Upon no completion of the processing and the outputof pixel data with regard to all the pixels (step S450: No), on theother hand, the processing flow goes back to step S100 in the flowchartof FIG. 4 to detect a defective pixel with regard to a target pixel of anext object pixel block.

The red light, the green light, and the blue light have different visualsensitivities. Different values are accordingly set to the thresholdvalues Vth(R), Vth(G), and Vth(B) for the red (R), green (B), and theblue (B) target pixels Rt, Bt, and Gt, as well as to the thresholdvalues Vjth1(R), Vjth1(G), and Vjth1(B) and to the threshold valuesVjth2(R), Vjth2(G), and Vjth2(B). Such setting enables the defectivepixel identification module 20 e to accurately identify whether each ofthe target pixels Rt, Bt, and Gt is a defective pixel according to thetype (color) of the color filter.

The pixel data output by the output unit 40 at step S240, S340, or S440is subjected to a required series of image processing performed by ahardware image processing circuit or by an image processing module ofthe software configuration. Typical examples of the image processinginclude white balance adjustment, gamma conversion, color correction,tone correction, contrast adjustment, and sharpness adjustment. Theprocessed pixel data may be displayed on a display unit, such as aliquid crystal panel or may be subjected to data compression accordingto a preset format and recorded in a recording medium, such as a flashmemory. The illustration and the detailed description of such processingare omitted in the specification hereof.

As described above, the imaging device 100 of the first embodiment isequipped with the defective pixel detector 20 to determine whether eachof the target pixels Rt, Gt, and Bt has a high potential or a relativelylow potential for recognition as a defective pixel in the imagingprocess. Based on the results of such determination, the defective pixeldetector 20 can successively and accurately identify whether each of thetarget pixels Rt, Gt, and Bt is a defective pixel, regardless of thetype of the defect as white defect or black defect. In the imagingdevice 100 of the first embodiment, in response to identification ofeach target pixel Rt, Gt, or Bt as a defective pixel by the defectivepixel detector 20, the pixel data correction unit 30 corrects the pixeldata of the target pixel Rt, Gt, or Bt identified as the defective pixeland the output unit 40 outputs the corrected pixel data.

B. Second Embodiment B1. Structure of Imaging Device

FIG. 9 schematically illustrates the structure of another imaging device100A in a second embodiment of the invention. The imaging device 100A ofthe second embodiment has an imaging assembly 10A of a three-plate typeincluding three image sensors 12R, 12G, and 12B, for example, CMOSsensors. The imaging assembly 10A has a light color separation opticalsystem, in addition to the zoom lens, the focusing lens, and theaperture mechanism included in the imaging assembly 10 of the firstembodiment. The light color separation optical system includes a prismarranged to separate the incident light entering via the zoom lens, thefocusing lens, and the aperture mechanism into three color lights red(R), green (G), and blue (B). The image sensors 12R, 12G, and 12Brespectively generate red (R) pixel data, green (G) pixel data, and blue(B) pixel data. Each of the image sensors 12R, 12G, and 12B has multiplelight receiving elements arrayed in a matrix (not shown), as in theimage sensor 12 of the first embodiment. In the imaging device 100A ofthe second embodiment, since the imaging assembly 10A has the lightcolor separation optical system, no color filter is set on each of thelight receiving elements in the respective image sensors 12R, 12G, and12B.

The imaging device 100A has three defective pixel detectors 20R, 20G,and 20B and three pixel data correction units 30R, 30G, and 30Bcorresponding to the respective image sensors 12R, 12G, and 12B, and anoutput unit 40A. Like the imaging device 100 of the first embodiment,the imaging device 100A of the second embodiment successively detectsdefective pixels (white defects and black defects) among multiple pixelsconstituting a taken image, corrects pixel data of the detecteddefective pixels, and outputs the corrected pixel data. The defectivepixel detectors 20R, 20G, and 20B, the pixel data correction units 30R,30G, and 30B, and the output unit 40A are configured by the hardware inthis embodiment, although some part of these constituents may beimplemented by the software configuration. The imaging device 100A alsoincludes a display unit constructed to display the taken image, forexample, a liquid crystal panel, and a recorder unit constructed tostore the taken image as pixel data in a recording medium, such as aflash memory, although not being specifically illustrated. The imagingdevice 100A further has a control unit 50A configured to include a CPU,a RAM, and a ROM and control the respective constituents of the imagingdevice 100A.

The defective pixel detector 20R includes a pixel data acquisitionmodule 20Ra, a first computation module 20Rb, a defective pixelcriterion setting module 20Rc, a second computation module 20Rd, and adefective pixel identification module 20Re. The defective pixel detector20G includes a pixel data acquisition module 20Ga, a first computationmodule 20Gb, a defective pixel criterion setting module 20Gc, a secondcomputation module 20Gd, and a defective pixel identification module20Ge. The defective pixel detector 20B includes a pixel data acquisitionmodule 20Ba, a first computation module 20Bb, a defective pixelcriterion setting module 20Bc, a second computation module 20Bd, and adefective pixel identification module 20Be. The pixel data acquisitionmodules 20Ra, 20Ga, and 20Ba respectively have buffers 20Rab, 20Gab, and20Bab.

The functions of the defective pixel detectors 20R, 20G, and 20B, thepixel data correction units 30R, 30G, and 30B, and the output unit 40Aare substantially equivalent to the functions of the defective pixeldetector 20, the pixel data correction unit 30, and the output unit 40in the first embodiment. The defective pixel detectors 20R, 20G, and 20Bcorrespond to the defective pixel detector of the invention.

B2. Defective Pixel Detection Process and Pixel Data Correction Process

The processing flow of the defective pixel detection process and thepixel data correction process performed in the second embodiment issimilar to the processing flow of the defective pixel detection processand the pixel data correction process performed in the first embodiment.The imaging device 100A of the second embodiment, however, includes thethree defective pixel detectors 20R, 20G, and 20B and the three pixeldata correction units 30R, 30G, and 30B corresponding to the respectiveimage sensors 12R, 12G, and 12B. Unlike the processing flow of the firstembodiment, the processing flow of the second embodiment does notperform the different series of processing with regard to the respectivecolors of the target pixels. The three defective pixel detectors 20R,20G, and 20B and the three pixel data correction units 30R, 30G, and 30Bsimultaneously perform an identical series of processing.

FIG. 10 is a flowchart showing the defective pixel detection process andthe pixel data correction process performed in the second embodiment.FIG. 11 schematically shows an arrangement of pixels in a 3×3 pixelblock. In this embodiment, a pixel located on the center of the pixelblock and encircled by the thick line represents a target pixel Pt as anobject of defective pixel detection. All surrounding pixels P1 to P8located in the neighborhood of the target pixel Pt are specified asperipheral pixels for defective pixel identification. As mentionedabove, the three defective pixel detectors 20R, 20G, and 20B haveidentical functions, and the three mage data correction units 30R, 30G,and 30B have identical functions. The description accordingly regardsthe defective pixel detection process and the pixel data correctionprocess for only red (R) pixels, while the explanation is omitted withregard to the defective pixel detection process and the pixel datacorrection process for green (G) pixels and blue (B) pixels. Theprocessing steps in the defective pixel detection process and the pixeldata correction process of the second embodiment identical with those inthe defective pixel detection process and the pixel data correctionprocess of the first embodiment are not specifically explained here.

Referring to the flowchart of FIG. 10, the pixel data acquisition module20Ra first obtains analog pixel data of each 3×3 pixel block as acurrent processing object out of analog pixel data output fromrespective light receiving elements of the image sensor 12R, performsanalog-to-digital (A-D) conversion of the obtained analog pixel data,stores the A-D converted pixel data as pixel values of the respectivepixels in the object pixel block into the buffer 20Rab, and sets acenter pixel located on the center of the object pixel block as a targetpixel Pt (step S500).

The subsequent series of processing of steps S510 to S560 is identicalwith the processing of steps S200 to S250 shown in the flowchart of FIG.5.

In this embodiment, absolute values Vabs1 corresponding to the firstabsolute values of the invention calculated at step S510 in theflowchart of FIG. 10 include the absolute value of the differencebetween the pixel values of the peripheral pixels P1 and P2, theabsolute value of the difference between the pixel values of theperipheral pixels P2 and P3, the absolute value of the differencebetween the pixel values of the peripheral pixels P3 and P5, theabsolute value of the difference between the pixel values of theperipheral pixels P5 and P8, the absolute value of the differencebetween the pixel values of the peripheral pixels P8 and P7, theabsolute value of the difference between the pixel values of theperipheral pixels P7 and P6, the absolute value of the differencebetween the pixel values of the peripheral pixels P6 and P4, theabsolute value of the difference between the pixel values of theperipheral pixels P4 and P5, and the absolute value of the differencebetween the pixel values of the peripheral pixels P2 and P7 (see FIG.11).

A fixed threshold value Vth corresponding to the preset threshold valueof the invention is used at step S520 in FIG. 10 with regard to red (R),green (G), and blue (B). Different threshold values Vth mayalternatively be used with regard to red (R), green (G), and blue (B),like the first embodiment.

A fixed threshold value Vjth1 corresponding to the first defective pixelcriterion of the invention is set to a defective pixel criterion Vjth atstep S521 in FIG. 10 with regard to red (R), green (G), and blue (B).Different threshold values Vjth1 may alternatively be set to thedefective pixel criterion Vjth with regard to red (R), green (G), andblue (B), like the first embodiment.

A fixed threshold value Vjth2 corresponding to the second defectivepixel criterion of the invention is set to the defective pixel criterionVjth at step S524 in FIG. 10 with regard to red (R), green (G), and blue(B). Different threshold values Vjth2 may alternatively be set to thedefective pixel criterion Vjth with regard to red (R), green (G), andblue (B), like the first embodiment.

As described above, the imaging device 100A of the second embodiment isequipped with the imaging assembly 10A including the three image sensors12R, 12G, and 12B and with the three defective pixel detectors 20R, 20G,and 20B provided corresponding to these three image sensors 12R, 12G,and 12B. Each of the defective pixel detectors 20R, 20G, and 20Bdetermines whether the target pixel Pt has a high potential or arelatively low potential for recognition as a defective pixel in theimaging process. Based on the results of such determination, thedefective pixel detectors 20R, 20G, and 20B can successively andaccurately identify whether each of the target pixels Pt is a defectivepixel, regardless of the type of the defect as white defect or blackdefect. In the imaging device 100A of the second embodiment, in responseto identification of the target pixel Pt as a defective pixel by thedefective pixel detector 20R, 20G, or 20B, the corresponding pixel datacorrection unit 30R, 30G, or 30B corrects the pixel data of the targetpixel Pt identified as the defective pixel and the output unit 40Aoutputs the corrected pixel data.

C. Other Aspects

The first and the second embodiments discussed above are to beconsidered in all aspects as illustrative and not restrictive. There maybe many modifications, changes, and alterations without departing fromthe scope or spirit of the main characteristics of the presentinvention. Some examples of possible modification are given below.

C1. Modified Example 1

In the procedure of the first embodiment, the defective pixel criterionsetting module 20 c sets the defective pixel criterion Vjth(R) for thered (R) target pixel Rt to the threshold value Vjth1(R) when at leastone of the multiple absolute values Vabs1(R) is not less than the presetthreshold value Vth(R), while setting the defective pixel criterionVjth(R) to the threshold value Vjth2(R) when all the multiple absolutevalues Vjth(R) are less than the preset threshold value Vth(R). Thisarrangement is, however, neither essential nor restrictive. In onemodified procedure, the defective pixel criterion setting module 20 cmay set the defective pixel criterion Vjth(R) to the threshold valueVjth1(R) when all the multiple absolute values Vabs1(R) are not lessthan the preset threshold value Vth(R), while setting the defectivepixel criterion Vjth(R) to the threshold value Vjth2(R) when at leastone of the multiple absolute values Vjth(R) is less than the presetthreshold value Vth(R). Such modification is also applicable to thegreen (G) target pixel Gt and the blue (B) target pixel Bt.

C2. Modified Example 2

In the procedure of the first embodiment, the defective pixelidentification module 20 e identifies the red (R) target pixel Rt as adefective pixel when all the multiple absolute values Vabs2(R) aregreater than the threshold value Vjth1(R), while identifying the targetpixel Rt as non-defective pixel when at least one of the multipleabsolute values Vabs2(R) is not greater than the threshold valueVjth1(R). This arrangement is, however, neither essential norrestrictive. In one modified procedure, the defective pixelidentification module 20 e may identify the target pixel Rt as adefective pixel when at least one of the multiple absolute valuesVabs2(R) is greater than the threshold value Vjth1(R), while identifyingthe target pixel Rt as non-defective pixel when all the multipleabsolute values Vabs2(R) are not greater than the threshold valueVjth1(R). Such modification is also applicable to the green (G) targetpixel Gt and the blue (B) target pixel Bt.

C3. Modified Example 3

In the first embodiment, the pixel data correction unit 30 calculatesthe simple average of the pixel data of the peripheral pixels fordefective pixel identification specified with regard to the target pixeland replaces the pixel data of the target pixel by the calculatedaverage to correct the pixel data of the target pixel. This arrangementis, however, neither essential nor restrictive. In one modification, thepixel data correction unit 30 may replace the pixel data of the targetpixel by a weighted average of the pixel data of the peripheral pixelsfor defective pixel identification, instead of the simple average of thepixel data of the peripheral pixels for defective pixel identification.Such modification is also applicable to the second embodiment.

C4. Modified Example 4

In the first embodiment, the pixel data acquisition module 20 a obtainspixel data of a 5×5 pixel block. In the second embodiment, the threepixel data acquisition modules 20Ra, 20Ga, and 20Ba respectively obtainpixel data of 3×3 pixel blocks. This arrangement is, however, neitheressential nor restrictive and may be modified arbitrarily to obtainpixel data of a target pixel and pixel data of surrounding pixelslocated in a neighborhood of the target pixel. For the high-speedexecution of the defective pixel detection process and the pixel datacorrection process, the smallest possible number of pixel data obtainedis preferable.

C5. Modified Example 5

In the first embodiment, different values are set to the threshold valueVth(R) for the red (R) target pixel Rt, to the threshold value Vth(G)for the green (G) target pixel Gt, and to the threshold value Vth(B) forthe blue (B) target pixel Bt. Such setting is, however, neitheressential nor restrictive. One identical value may alternatively be setto all the threshold values Vth(R), Vth(G), and Vth(B).

In the first embodiment, different values are set to the threshold valueVjth1(R) for the red (R) target pixel Rt, to the threshold valueVjth1(G) for the green (G) target pixel Gt, and to the threshold valueVjth1(B) for the blue (B) target pixel Bt. Such setting is, however,neither essential nor restrictive. One identical value may alternativelybe set to all the threshold values Vjth1(R), Vjth1(G), and Vjth1(B).

In the first embodiment, different values are set to the threshold valueVjth2(R) for the red (R) target pixel Rt, to the threshold valueVjth2(G) for the green (G) target pixel Gt, and to the threshold valueVjth2(B) for the blue (B) target pixel Bt. Such setting is, however,neither essential nor restrictive. One identical value may alternativelybe set to all the threshold values Vjth2(R), Vjth2(G), and Vjth2(B).

C6. Modified Example 6

In the first embodiment and the second embodiment described above, thethreshold values Vth(R), Vth(G), and Vth(B), the threshold valuesVjth1(R), Vjth1(G), and Vjth1(B), and the threshold values Vjth2(R),Vjth2(G), and Vjth2(B) are all fixed values. Such setting is, however,neither essential nor restrictive. These threshold values mayalternatively be varied according to the magnitudes of the pixel valuesof the peripheral pixels for defective pixel identification. The largerpixel values of the peripheral pixels for defective pixel identificationrepresenting the higher luminance increase the potential for recognitionof the target pixel as a black defect. The smaller pixel values of theperipheral pixels for defective pixel identification representing thelower luminance increase the potential for recognition of the targetpixel as a white defect. In other words, the larger pixel values of theperipheral pixels for defective pixel identification representing thehigher luminance decrease the potential for recognition of the targetpixel as a white defect. The smaller pixel values of the peripheralpixels for defective pixel identification representing the lowerluminance decrease the potential for recognition of the target pixel asa black defect.

One concrete procedure of varying the above threshold values accordingto the magnitudes of the pixel values of the peripheral pixels fordefective pixel identification may set each of these threshold valuesbased on a simple average or a weighted average of the pixel values ofthe peripheral pixels for defective pixel identification. In this case,there may be a linear or non-linear relation between each of thethreshold values and the simple average or the weighted average of thepixel values of the peripheral pixels for defective pixelidentification. Each of the threshold values may be calculated accordingto a predetermined operation based on this linear or non-linearrelation. Each of the threshold values may otherwise be set withreference to a table recording the linear or non-linear relation betweeneach of the threshold values and the simple average or the weightedaverage of the pixel values of the peripheral pixels for defective pixelidentification.

C7. Modified Example 7

The first embodiment adopts the CMOS sensor for the image sensor 12.This is, however, not restrictive. In another example, a CCD may beadopted for the image sensor 12. In this latter application, any ofvarious types of CCDs may be used for the image sensor 12, for example,single-shot type (single-plate CCD), multi-shot type, scanner type, and3CCD type. Such modification is also applicable to the secondembodiment.

1. A defective pixel detector mounted on an imaging device andconstructed to detect a defective pixel among multiple pixelsconstituting an image taken with the imaging device, the defective pixeldetector comprising: a pixel data acquisition module configured tosuccessively obtain pixel data representing a pixel value of a targetpixel set as an object of defective pixel detection and pixel datarepresenting pixel values of plural surrounding peripheral pixelslocated in a neighborhood of the target pixel; a first operation moduleconfigured to calculate absolute values of differences between pixelvalues of multiple specific peripheral pixels selected among the pluralsurrounding pixels, as first absolute values; a defective pixelcriterion setting module configured to set a defective pixel criterion,which is used in subsequent identification of whether the target pixelis a defective pixel, based on differences between the multiple firstabsolute values and a preset threshold value; a second operation moduleconfigured to calculate absolute values of differences between the pixelvalue of the target pixel and the pixel values of the multiple specificperipheral pixels, as second absolute values; and a defective pixelidentification module configured to identify whether the target pixel isa defective pixel, based on the multiple second absolute values and theset defective pixel criterion.
 2. The defective pixel detector inaccordance with claim 1, wherein the defective pixel criterion settingmodule sets the defective pixel criterion to a first defective pixelcriterion when the multiple first absolute values are respectively notless than the preset threshold value, while setting the defective pixelcriterion to a second defective pixel criterion, which is smaller thanthe first defective pixel criterion, when the multiple first absolutevalues are respectively less than the preset threshold value.
 3. Thedefective pixel detector in accordance with claim 2, wherein thedefective pixel criterion setting module sets at least one of the firstdefective pixel criterion and the second defective pixel criterion,based on the pixel values of the multiple specific peripheral pixels. 4.The defective pixel detector in accordance with claim 1, wherein thedefective pixel identification module identifies the target pixel as adefective pixel when the multiple second absolute values arerespectively greater than the defective pixel criterion.
 5. Thedefective pixel detector in accordance with claim 1, wherein the imagingdevice is equipped with an image sensor including multiple lightreceiving elements provided corresponding to the multiple pixels andarranged to respectively output pixel data representing pixel values ofthe corresponding pixels, multiple different types of color filtersdesigned to transmit different color lights are set in a predeterminedarrangement on the multiple light receiving elements, and the multiplespecific peripheral pixels correspond to specific light receivingelements having a selected type of color filters identical with a colorfilter set on a light receiving element corresponding to the targetpixel.
 6. The defective pixel detector in accordance with claim 5,wherein the preset threshold value is set for each of the multipledifferent types of color filters.
 7. The defective pixel detector inaccordance with claim 5, wherein the defective pixel criterion is setfor each of the multiple different types of color filters.
 8. An imagingdevice, comprising: an imaging assembly equipped with an image sensorincluding multiple light receiving elements provided corresponding tomultiple pixels constituting a taken image and arranged to respectivelyoutput pixel data representing pixel values of the corresponding pixels;a defective pixel detector configured to detect a defective pixel amongthe multiple pixels, based on the pixel data respectively output fromthe multiple light receiving elements; and a pixel data correction unitconfigured to correct pixel data representing a pixel value of thedetected defective pixel, wherein the defective pixel detector is thedefective pixel detector in accordance with any one of claims 1 through7.
 9. The imaging device in accordance with claim 8, wherein the pixeldata correction unit replaces the pixel data of the detected defectivepixel by an average of the pixel values of the multiple specificperipheral pixels.
 10. A defective pixel detection method configured todetect a defective pixel among multiple pixels constituting an imagetaken with an imaging device, the defective pixel detection methodcomprising: successively obtaining pixel data representing a pixel valueof a target pixel set as an object of defective pixel detection andpixel data representing pixel values of plural surrounding peripheralpixels located in a neighborhood of the target pixel; calculatingabsolute values of differences between pixel values of multiple specificperipheral pixels selected among the plural surrounding pixels, as firstabsolute values; setting a defective pixel criterion, which is used insubsequent identification of whether the target pixel is a defectivepixel, based on differences between the multiple first absolute valuesand a preset threshold value; calculating absolute values of differencesbetween the pixel value of the target pixel and the pixel values of themultiple specific peripheral pixels, as second absolute values; andidentifying whether the target pixel is a defective pixel, based on themultiple second absolute values and the set defective pixel criterion.